Method of detecting human and/or animal motion and performing mobile disinfection

ABSTRACT

Implementations of the disclosed subject matter provide a method of moving a mobile robot within an area. The movement of the mobile robot and the emission of ultraviolet (UV) light may be stopped when a human and/or animal is determined to be within the area. Using at least one sensor, the method may be determine whether there is at least one of human identification, animal identification, motion, heat, and/or sound within the area for a predetermined period of time. When there is no human identification, animal identification, motion, heat, and/or sound within the predetermined period of time, UV light may be emitted and the drive system may be controlled to move the mobile robot within the area. When there is at least one of human identification, motion, heat, and/or sound within the predetermined period of time, a light source may be controlled to prohibit the emission of UV light.

BACKGROUND

Mobile devices, such as mobile robots, can be operated so as todisinfect indoor areas, such as a room that has surfaces contaminatedwith bacteria, viruses, or other pathogens. When such robots areremotely operated, it is difficult for the remote operator to determinewhen a person enters an area that is being disinfected. Similarly, whensuch robots are autonomously operated, it is difficult for the robot todetermine when a person enters an area that is being disinfected. Somedisinfection modalities can be harmful to humans and/or animals.

BRIEF SUMMARY

According to an implementation of the disclosed subject matter, a methodmay be provided that includes moving, using a drive system, a mobilerobot within an area, and emitting, using a light source of the mobilerobot, ultraviolet (UV) light to disinfect at least a portion of thearea. While emitting the UV light, the method may determine whetherthere is a human and/or an animal within the area using at least onesensor. The movement of the mobile robot within the area may be stoppedby controlling the drive system and stopping the emission of the UVlight by controlling the UV light source when the human and/or animal isdetermined to be within the area. The method may include determining,using the at least one sensor, whether there is at least one of humanand/or animal identification, motion, heat, and/or sound within the areafor a predetermined period of time. When the at least one sensordetermines that there is no human and/or animal identification, motion,heat, and sound within the predetermined period of time, the lightsource may be controlled to emit UV light and the drive system may becontrolled to move the mobile robot within the area. When the at leastone sensor determines that there is at least one of human and/or animalidentification, motion, heat, and/or sound within the predeterminedperiod of time, the light source may be controlled to prohibit theemission of UV light.

Additional features, advantages, and implementations of the disclosedsubject matter may be set forth or apparent from consideration of thefollowing detailed description, drawings, and claims. Moreover, it is tobe understood that both the foregoing summary and the following detaileddescription are illustrative and are intended to provide furtherexplanation without limiting the scope of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the disclosed subject matter, are incorporated in andconstitute a part of this specification. The drawings also illustrateimplementations of the disclosed subject matter and together with thedetailed description serve to explain the principles of implementationsof the disclosed subject matter. No attempt is made to show structuraldetails in more detail than may be necessary for a fundamentalunderstanding of the disclosed subject matter and various ways in whichit may be practiced.

FIG. 1 shows an example method of controlling the movement of a mobilerobot within a predetermined area, outputting ultraviolet (UV) lighthaving a dosage level to disinfect the predetermined area, and stoppingthe output of UV light when a human and/or an animal is detectedaccording to an implementation of the disclosed subject matter.

FIG. 2 shows that the example method of FIG. 1 may include determiningwhen the human and/or animal is present in the area according to animplementation of the disclosed subject matter.

FIG. 3 shows that the example method of FIG. 1 may include a method ofdetecting a door opening within the area and stopping the movement andthe UV light output by the robot based on the detection according to animplementation of the disclosed subject matter.

FIGS. 4-6 show a plurality of external views of the mobile robot havingsensors and a light source to output UV light according toimplementations of the disclosed subject matter.

FIG. 7A shows an example thermal image captured by at least one sensorof the mobile robot where no human and/or animal is present according toan implementation of the disclosed subject matter.

FIG. 7B shows an example thermal image captured by at least one sensorof the mobile robot where a human and/or an animal is present accordingto an implementation of the disclosed subject matter.

FIG. 7C shows an example thermal image captured by at least one sensorof the mobile robot where the human and/or animal detected in FIG. 7B ispresent with different clothing according to an implementation of thedisclosed subject matter.

FIG. 8 shows a door sensor and a mobile robot having at least one sensorto monitor the opening of the door according to an implementation of thedisclosed subject matter.

FIG. 9 shows an example of a path of the mobile robot to apply a dosageof UV light in an area and map non-human locations according to animplementation of the disclosed subject matter.

FIG. 10 shows an example configuration of the mobile robot of FIGS. 4-6according to an implementation of the disclosed subject matter.

FIG. 11 shows a network configuration which may include a plurality ofmobile robots according to implementations of the disclosed subjectmatter.

DETAILED DESCRIPTION

Implementations of the disclosed subject matter provide a mobile robotthat may move about a predetermined area and output ultraviolet (UV)light to disinfect the predetermined area. The mobile robot may be usedas part of a regular cleaning cycle of a room, building, airplane,school, or the like, and may prevent and/or reduce the spread ofinfectious diseases, viruses, bacteria, and other types of harmfulorganic microorganisms in the environment by breaking down theirDNA-structure with UV light.

Implementations of the disclosed subject matter provide methods ofcontrolling the operation of the mobile robot to protect humans and/oranimals from the UV light output by the mobile robot. Sensors may beplaced at the one or more entrances to an area to be disinfected. Inhospitals and/or other buildings, there may be a need to disinfect largeareas, which may have a plurality of entrances. Walls and otherobstacles of the building and/or room may cause interference to thecommunication between the sensors disposed near an entrance to a roomand a mobile robot. Implementations of the disclosed subject matterprovide a mobile robot with one or more sensors that may be used todetermine if a human and/or an animal is within an area beingdisinfected with UV light. For example, the one or more sensors mayinclude an image sensor, RGB (red green blue) camera, thermal camera,geometrical sensors, a microphone, and the like.

In some implementations, the sensor (e.g., an RGB camera, an imagesensor, or the like) may determine if a human and/or an animal is withinthe captured image. Once a human and/or an animal has been detected, themobile robot may be controlled so as to stop its movement and to stopthe output of UV light from the light source for a predetermined periodof time so that the human and/or animal will not be harmed and that themobile robot may more accurately determine if a human and/or an animalis present in the area. In some implementations, when the human and/oranimal is detected, the mobile robot may be controlled to stop theoutput of UV light from the light source for a predetermined period oftime, and may be controlled to move to a predetermined portion of thearea. The mobile robot may provide an audio and/or visual alert for thestoppage. That is, while the mobile robot has stopped and is no longeroutputting UV light, it may use the one or more sensors to determinewhether a human and/or animal is present within the area.

If an object initially detected as human and/or animal is confirmedduring the predetermined period of time to be “non-human” and/or“non-animal,” the non-human and/or non-animal object may be identifiedon a map and stored in memory. That is, the next time that the sameobject is detected as a human and/or an animal in the same area of themap, it may not be triggered as a potential human and/or potentialanimal by the one or more sensors of the robot. In some implementations,this may be disabled to provide additional safety, so that each objectdetected as a potential human and/or potential may stop the operationsof the robot so that the object may be confirmed as being human and/oranimal, or not.

If a thermal camera is used by the mobile robot to initially determinewhether a human and/or animal is within the room, there may be morefalse positives with this type of sensor. Stopping the mobile robot andthe output of UV light may be important for human and/or animal safety.For example, hospital personnel may wear protective clothing, and thethermal camera may have difficulty detecting them as humans. In anotherexample, light bulbs may be in the shape of human heads, and may lead tofalse positives. The mobile robot may stop and refrain from outputtingUV light to determine whether the detected object is a human and/or ananimal by using one or more sensors.

In implementations of the disclosed subject matter, the at least onesensor of the mobile robot may include geometric sensors, which may beused by the robot to determine its position within the area and/or on amap. When the mobile robot initially determines that an object may be ahuman and/or an animal, and no such object is on the map, the mobilerobot may stop moving and outputting UV light, and may determine whetherthe object is a human and/or an animal or not over the predeterminedperiod of time.

In implementations of the disclosed subject matter, the mobile robot maydetermine whether a human and/or an animal is within the area after aninitial detection by stopping the movement of the mobile robot and theoutput of UV light, and using one or more sensors to determine whether ahuman and/or an animal is present in the area over a predeterminedperiod of time. This may provide additional safety to humans and/oranimals within the area while the mobile robot performs disinfectionoperations.

FIG. 1 shows an example method 10 of controlling the movement of amobile robot within a predetermined area, outputting ultraviolet (UV)light having a dosage level to disinfect the predetermined area, andstopping the movement and outputting of UV light when a human and/or ananimal is detected according to an implementation of the disclosedsubject matter. At operation 12, a mobile robot (e.g., mobile robot 100shown in FIGS. 4-6 and 8-11) may move within an area (e.g., 300 as shownin FIGS. 8-9) using a drive system (e.g. drive system 108 shown in FIG.10). At operation 14, a light source (e.g., light source 104 shown inFIGS. 4, 6, 8, and 9 and described below) of the mobile robot may emitultraviolet (UV) light to disinfect at least a portion of the area.

At operation 16, while emitting the UV light from the light source ofthe mobile robot, the controller (e.g., controller 114 shown in FIG. 10)and/or the at least one sensor may be used to determine whether there isa human and/or an animal within the area using at least one sensor(e.g., sensor 102 and/or sensor 106 and/or microphone 103 shown in shownin FIGS. 4-6, 8, and 10).

In some implementations, the determining whether the human and/or animalis within the area in operation 16 may include capturing an image, usingat least one sensor, to determine whether the human and/or animal iswithin the area. For example, the sensor 102 and/or 106 of the mobilerobot 100 that may capture the image. The sensor 102, 106 and/or thecontroller 114 may determine whether there is a human and/or an animalpresent within the captured image by using image and/or patternrecognition.

In some implementations, the determining whether the human and/or theanimal is within the area in operation 16 may include capturing athermal image by using a thermal sensor. The thermal sensor may besensor 102 and/or 106 of the mobile robot 100. The sensor 102, 106and/or the controller 114 may determine whether there is a human and/oran animal present within the captured thermal image by using, forexample, pattern recognition.

At operation 18, the controller may stop the movement of the mobilerobot within the area by controlling the drive system (e.g., drivesystem 108 shown in FIG. 10) and stopping the emission of the UV lightby controlling the UV light source (e.g., light source 104 shown inFIGS. 4, 6, 8, and 9) when the human and/or the animal is determined tobe within the area.

At operation 20, the at least one sensor may be used to determinewhether there is at least one of human and/or animal identification,motion, heat, and/or sound within the area for a predetermined period oftime. That is, the at least one sensor and/or the controller may be usedto identify whether a human and/or an animal is present within acaptured image, whether there is motion within the area which mayindicate that a human and/or an animal is present based on the patternof motion, whether a human and/or an animal is present within a thermalimage based on emission profile of the image, and/or whether soundcaptured by the microphone indicates that a human and/or an animal ispresent within the area (e.g., detection of voice, animal noises,footsteps, breathing, or the like).

In implementations of the disclosed subject matter, a human and/or ananimal may be detected by a thermal camera and/or infrared camera, whichmay be sensor 102, 106. For example, the mobile robot having the thermalcamera and/or infrared camera may determine the heat of an object and/orone or more portions of the object within the view of the camera. If theheat is around 36° C., the mobile robot may determine that a human issuspected as appearing in the image captured by the camera. In someimplementations, when a human and/or animal is located a predetermineddistance or more from the camera, the heat information may be capturedby one or more pixels of the camera.

Implementations of the disclosed subject matter provide a mobile robotthat may move about a predetermined area and output ultraviolet (UV)light to disinfect the predetermined area. The mobile robot may be usedas part of a regular cleaning cycle of a room, building, airplane,school, or the like, and may prevent and/or reduce the spread ofinfectious diseases, viruses, bacteria, and other types of harmfulorganic microorganisms in the environment by breaking down theirDNA-structure with UV light.

Implementations of the disclosed subject matter provide methods ofcontrolling the operation of the mobile robot to protect humans and/oranimals from the UV light output by the mobile robot. Sensors may beplaced at the one or more entrances to an area to be disinfected. Inhospitals and/or other buildings, there may be a need to disinfect largeareas, which may have a plurality of entrances. Walls and otherobstacles of the building and/or room may cause interference to thecommunication between the sensors disposed near an entrance to a roomand a mobile robot. Implementations of the disclosed subject matterprovide a mobile robot with one or more sensors that may be used todetermine if a human and/or an animal is within an area beingdisinfected with UV light. For example, the one or more sensors mayinclude an image sensor, RGB (red green blue) camera, thermal camera,geometrical sensors, microphone, and the like.

In some implementations, the sensor (e.g., an RGB camera, an imagesensor, or the like) may determine if a human and/or an animal is withinthe captured image. Once a human and/or an animal has been detected, themobile robot may be controlled so as to stop its movement and to stopthe output of UV light from the light source for a predetermined periodof time so that the human and/or animal will not be harmed and that themobile robot may more accurately determine if a human and/or an animalis present in the area. In some implementations, when the human and/oranimal is detected, the mobile robot may be controlled to stop theoutput of UV light from the light source for a predetermined period oftime, and may be controlled to move to a predetermined portion of thearea. The mobile robot may provide an audio and/or visual alert for thestoppage. That is, while the mobile robot has stopped and is no longeroutputting UV light, it may use the one or more sensors to determinewhether a human and/or an animal is present within the area.

If an object initially detected as human and/or animal is confirmedduring the predetermined period of time to be “non-human” or“non-animal,” the non-human and/or non-animal object may be identifiedon a map and stored in memory. That is, the next time that the sameobject is detected as a human and/or an animal in the same area of themap, it may not be triggered as a potential human and/or potentialanimal by the one or more sensors of the robot. In some implementations,this may be disabled to provide additional safety, so that each objectdetected as a potential human and/or potential animal may stop theoperations of the robot so that the object may be confirmed as being ahuman and/or animal, or not.

If a thermal camera is used by the mobile robot to initially determinewhether a human and/or an animal is within the room, there may be morefalse positives with this type of sensor. Stopping the mobile robot andthe output of UV light may be important for human and/or animal safety.For example, hospital personnel may wear protective clothing, and thethermal camera may have difficulty detecting them as humans and/oranimals. In another example, light bulbs may be in the shape of humanheads, and may lead to false positives. The mobile robot may stop andrefrain from outputting UV light to determine whether the detectedobject is a human and/or an animal by using one or more sensors.

In implementations of the disclosed subject matter, the at least onesensor of the mobile robot may include geometric sensors, which may beused by the robot to determine its position within the area and/or on amap. When the mobile robot initially determines that an object may be ahuman and/or an animal, and no such object is on the map, the mobilerobot may stop moving and outputting UV light, and may determine whetherthe object is a human and/or animal or not over the predetermined periodof time.

In implementations of the disclosed subject matter, the mobile robot maydetermine whether a human and/or animal is within the area after aninitial detection by stopping the movement of the mobile robot and theoutput of UV light, and using one or more sensors to determine whether ahuman and/or an animal is present in the area over a predeterminedperiod of time. This may provide additional safety to humans and/oranimals within the area while the mobile robot performs disinfectionoperations.

FIG. 1 shows an example method 10 of controlling the movement of amobile robot within a predetermined area, outputting ultraviolet (UV)light having a dosage level to disinfect the predetermined area, andstopping the movement and outputting of UV light when a human and/or ananimal is detected according to an implementation of the disclosedsubject matter. At operation 12, a mobile robot (e.g., mobile robot 100shown in FIGS. 4-6 and 8-11) may move within an area (e.g., 300 as shownin FIGS. 8-9) using a drive system (e.g. drive system 108 shown in FIG.10). At operation 14, a light source (e.g., light source 104 shown inFIGS. 4, 6, 8, and 9 and described below) of the mobile robot may emitultraviolet (UV) light to disinfect at least a portion of the area.

At operation 16, while emitting the UV light from the light source ofthe mobile robot, the controller (e.g., controller 114 shown in FIG. 10)and/or the at least one sensor may be used to determine whether there isa human and/or an animal within the area using at least one sensor(e.g., sensor 102 and/or sensor 106 shown in FIGS. 4-6 and 8, and/ormicrophone 109 shown in FIG. 10, which may be one of the sensors).

In some implementations, the determining whether the human and/or animalis within the area in operation 16 may include capturing an image (e.g.,image 252, 258, 266 shown in FIGS. 7A-7C and described below) using atleast one sensor, to determine whether the human and/or animal is withinthe area. For example, the sensor 102 and/or 106 of the mobile robot 100that may capture the image. The sensor 102, 106, the controller 114, theserver 140, and/or the remote platform 160 may determine whether thereis a human and/or an animal present within the captured image by usingimage and/or pattern recognition.

In some implementations, the determining whether the human and/or animalis within the area in operation 16 may include capturing a thermal image(e.g., image 250, 256, 264 shown in FIGS. 7A-7C and described below) byusing a thermal sensor. The thermal sensor may be sensor 102 and/or 106of the mobile robot 100. The sensor 102, 106, the controller 114, theserver 140, and/or the remote platform 160 may determine whether thereis a human and/or an animal present within the captured thermal image byusing, for example, pattern recognition.

At operation 18, the controller may stop the movement of the mobilerobot within the area by controlling the drive system (e.g., drivesystem 108 shown in FIG. 10) and stopping the emission of the UV lightby controlling the UV light source (e.g., light source 104 shown inFIGS. 4, 6, 8, and 9) when the human and/or animal is determined to bewithin the area. In some implementations, when the mobile robot 100 isbeing remotely operated (e.g., from commands received from the server140 and/or remote platform 160 shown in FIG. 11), the remote operatormay stop the movement of the mobile robot and/or operation of the UVlight source when the remote operator determines that a human and/oranimal is within the area. In some implementations, the controller maystop the movement of the mobile robot and/or the emission of UV lightfrom the UV light source when a voice command is received by the mobilerobot via a microphone (e.g., microphone 103 shown in FIG. 10). In someimplementations, the controller may stop the movement of the mobilerobot and/or the emission of UV light from the UV light source when stopbutton 112 (shown in FIG. 4) is selected.

At operation 20, the at least one sensor may be used to determinewhether there is at least one of human and/or animal identification,motion, heat, and/or sound within the area for a predetermined period oftime. That is, the at least one sensor and/or the controller may be usedto identify whether a human and/or an animal is present within acaptured image, whether there is motion within the area which mayindicate that a human and/or an animal is present based on the patternof motion, whether a human and/or an animal is present within a thermalimage based on emission profile of the image, and/or whether soundcaptured by the microphone indicates that a human and/or an animal ispresent within the area (e.g., detection of voice, animal noises,footsteps, breathing, or the like).

At operation 22, when the at least one sensor determines that there isno human and/or animal identification, motion, heat, and/or sound withinthe predetermined period of time, the light source may be controlled toemit UV light and the drive system to move the mobile robot within thearea. For example, the controller 114 shown in FIG. 10 may control thelight source 104 and/or the drive system 108 of the mobile robot to moveabout path 306 in area 300 shown in FIG. 9 to disinfect the area,including beds 312, sink 308, 4-hook IV stands 314, and the like whilethere are no humans and/or animals present in the area.

In some implementations, when the mobile robot 100 is being remotelyoperated (e.g., from commands received from the server 140 and/or remoteplatform 160 shown in FIG. 11), the remote operator may control themovement of the mobile robot and/or the operation of the UV light sourcewhen the remote operator determines that a human and/or animal is notwithin the area.

At operation 24, when the at least one sensor determines that there isat least one of human and/or animal identification, motion, heat, and/orsound within the predetermined period of time, the light source may becontrolled to prohibit the emission of UV light. For example, thecontroller 114 shown in FIG. 10 may control the light source 104 toprohibit the emission of UV light. In some implementations, thecontroller may prohibit the movement of the mobile robot and/or theemission of UV light from the UV light source when a voice command isreceived by the mobile robot via a microphone (e.g., microphone 103shown in FIG. 10). In some implementations, the controller may prohibitthe movement of the mobile robot and/or the emission of UV light fromthe UV light source when stop button 112 (shown in FIG. 4) is selected.

In some implementations, a notification may be output that the humanand/or animal is determined to be in the area, the movement of themobile robot is being stopped, and/or the emission of the UV light fromthe light source is stopped. The output notification may be an audiblenotification and/or a visual notification. For example, the speaker 107(shown in FIG. 10) of the mobile robot 100 may output the audiblenotification that the human and/or animal is determined to be in thearea, the movement of the mobile robot is being stopped, and/or theemission of the UV light from the light source is stopped. The userinterface 110 (shown in FIG. 10) of the mobile robot 100 may include adisplay to display the visual notification that the human and/or animalis determined to be in the area, the movement of the mobile robot isbeing stopped, and/or the emission of the UV light from the light sourceis stopped.

In some implementations, the notification may be transmitted using acommunications interface of the mobile robot via a communicationsnetwork. For example, the notification may be transmitted using thenetwork interface 116 (shown in FIG. 10) of the mobile robot 100 via thenetwork 130 to the server 140, database 150, and/or remote platform 160shown in FIG. 11.

In some implementations, the location of the detected human as anon-human and/or the detected animal as non-animal on a map of the areain a memory that is communicatively coupled to the controller based on asignal received by a communications interface of the mobile robot via acommunications network or an input received from an interface of themobile robot. For example, operation 16 may determine that there is ahuman present in the area, but operation 20 may determine that there isno human (i.e., the object originally detected is human is non-human) inthe area based on the lack of a human identification, motion detection,the heat, and/or sound signals generated by the at least one sensor. Thelocation of the non-human and/or non-animal in the area may be stored inthe fixed storage 120 (shown in FIG. 10) and/or the memory 118 of themobile robot 100, and/or at the server 140, database 150, and/or remoteplatform 160 as shown in FIG. 11. The location of the non-human and/ornon-animal may be determined by at least one of the sensor 102 and/or106 of the mobile robot 100.

As discussed above, operations 16, 18, and/or 20 may use sensors, 102,106 to determine whether a human and/or animal is present within thearea. FIG. 7A shows an example thermal image 250 that may be captured byat least one sensor 102, 106 of the mobile robot 100 where no humanand/or animal is present according to an implementation of the disclosedsubject matter. Image 252 may be a camera image captured by the at leastone sensor 102, 106, which shows that no human and/or animal is present.Although the thermal image 250 includes areas 254 of high thermalenergy, the shape of areas 254 may be determined by controller 114(shown in FIG. 10), and/or server 140 and/or remote platform 160 of FIG.11 as being non-human, based on the pattern of areas 254. The image 252may be used to confirm that a human and/or an animal is not present inthe thermal image 250.

FIG. 7B shows an example thermal image 256 captured by at least onesensor 102, 106 of the mobile robot where a human 259 is presentaccording to an implementation of the disclosed subject matter. Image258 may be a camera image captured by the at least one sensor 102, 106,which shows that the human 259 is present. The thermal image 256 mayinclude areas 260 and 262 which may have thermal energy levels greaterthan a predetermined amount that are in a shape of a human as determinedby controller 114 (shown in FIG. 10), and/or server 140 and/or remoteplatform 160 of FIG. 11. The pattern of areas 254 may be confirmed to behuman based on the image 258.

FIG. 7C shows an example thermal image 264 captured by at least onesensor 102, 106 of the mobile robot where the human detected in FIG. 7Bis present with different clothing (e.g., human 267) according to animplementation of the disclosed subject matter. Image 266 may be acamera image captured by the at least one sensor 102, 106, which showsthat the human 267 is present, with different clothing from human 259shown in FIG. 7B. The thermal image 264 may include areas 268 and 270which may have thermal energy levels greater than a predetermined amountthat are in a shape of a human, as determined by controller 114 (shownin FIG. 10), and/or server 140 and/or remote platform 160 of FIG. 11 asbeing non-human, based on the pattern of areas 254. The shape of theareas 268 and 270 may be different from that of areas 260 and 262 ofFIG. 7B, but may be identified as being a human shape, which may beverified using, for example, image 266.

That is, a controller (e.g., controller 114 shown in FIG. 10, and/orserver 140 and/or remote platform 160 shown in FIG. 11) and/or thesensor 102, 106 may perform human and/or animal detection based onsignals received from the one or more sensor (e.g., the thermal camera,the RGB camera, and the like). For example, if the detected shape of theobject resembles a human and/or an animal, the object may be determinedto be a human and/or an animal.

In some implementations, motion in the area may be initially determinedby the sensor 102, 106 as being a human and/or an animal. Although themotion may not necessarily be a human and/or an animal, when thedetermination that the motion is from a human and/or an animal, themobile robot may stop the output of UV light by the light source toprevent harm to any human and/or animal, and use the sensors 102, 106and/or microphone 103 over a predetermined period of time to moreaccurately determine whether the human and/or animal is within the area.

In some implementations, humans and/or animals may be detected withinthe area by using YOLO (You Only Look Once), a single neural network isapplied to a full image. The network divides the image into regions, andpredicts bounding boxes and probabilities for each region. Thesebounding boxes may be weighted by the predicted probabilities. YOLO hasseveral advantages over typical classifier-based systems. YOLO may usethe whole image at test time, so its predictions may be based on theglobal context in the image. Yolo may also make predictions with asingle network evaluation, unlike other systems which require thousandsfor a single image.

In some implementations, humans and/or animals may be detected using SSD(Single Shot Multibox Detection), which is a method for detectingobjects in images using a single deep neural network. SSD may discretizethe output space of bounding boxes into a set of default boxes overdifferent aspect ratios and scales per feature map location. Atprediction time, the network may generate scores for the presence ofeach object category in each default box, and may produce adjustments tothe box to better match the object shape. The network may combinepredictions from multiple feature maps with different resolutions tonaturally handle objects of various sizes. SSD may be simple relative totraditional methods that require object proposals, as SSD eliminatesproposal generation and subsequent pixel or feature resampling stage andencapsulates all computation in a single network. SSD may be easier totrain and may have better accuracy than single stage method, even with asmaller input image size.

The YOLO, SSD, and other human and/or animal detection methods may beused, for example, when the mobile robot is in a static position (i.e.,not moving) the mobile robot is static.

In some implementations, humans and/or animal may be detected in the bythe mobile robot based on sound received by the microphone (e.g.,microphone 103 shown in FIGS. 4-6, 8, and 10). For example, VoiceActivity Detection (VAD) may be used (e.g., by the controller 114 shownin FIG. 10, and/or the server 140 and/or the remote platform 160 shownin FIG. 11). VAD may be used determine if speech and/or noise from ahuman and/or animal is present or not in the area, even when there isbackground noise. VAD may break an audio signal (e.g., as received bythe microphone 103) into frames, extract features from each frame, traina classifier on a known set of speech and silence frames, and classifyunseen frames as speech or silence.

Implementations of the disclosed subject matter may combine two or moreof the methods disclosed above (e.g., thermal imaging, YOLO, SSD, VAD,and the like) to increase the robustness of the system. For example, toprevent a human and/or an animal from being harmed by the output of UVlight, the mobile robot may be stopped and the output of UV light may bestopped if motion is initially detected. The mobile robot may use thesensors 102, 106 and/or microphone 103 for a predetermined period oftime to determine if a human and/or an animal is within the area. If ahuman and/or animal is not detected within the predetermined time, themobile robot may output UV light and continue to disinfect the area. Ifa human and/or an animal is detected, the mobile robot may output anotification and/or instructions requesting that the human and/or animalleave the area so that disinfection may resume.

In some implementations, such as at operation 22 shown in FIG. 1, whenthe at least one sensor determines that there is no human and/or animalidentification, motion, heat, and/or sound within the predeterminedperiod of time, the light source may be controlled to emit UV light andthe drive system to move the mobile robot within the area. For example,the controller 114 shown in FIG. 10 may control the light source 104and/or the drive system 108 of the mobile robot to move about path 306in area 300 shown in FIG. 9 to disinfect the area, including beds 312,sink 308, 4-hook IV stands 314, and the like while there are no humansand/or animals present in the area.

At operation 24 shown in FIG. 1, when the at least one sensor determinesthat there is at least one of human and/or animal identification,motion, heat, and/or sound within the predetermined period of time, thelight source may be controlled to prohibit the emission of UV light. Forexample, the controller 114 shown in FIG. 10 may control the lightsource 104 to prohibit the emission of UV light.

In some implementations, a notification may be output that the humanand/or animal is determined to be in the area, the movement of themobile robot is being stopped, and/or the emission of the UV light fromthe light source is stopped. The output notification may be an audiblenotification and/or a visual notification. For example, the speaker 107(shown in FIG. 10) of the mobile robot 100 may output the audiblenotification that the human and/or animal is determined to be in thearea, the movement of the mobile robot is being stopped, and/or theemission of the UV light from the light source is stopped. The userinterface 110 (shown in FIG. 10) of the mobile robot 100 may include adisplay to display the visual notification that the human and/or animalis determined to be in the area, the movement of the mobile robot isbeing stopped, and/or the emission of the UV light from the light sourceis stopped.

In some implementations, the notification may be transmitted using acommunications interface of the mobile robot via a communicationsnetwork. For example, the notification may be transmitted using thenetwork interface 116 (shown in FIG. 10) of the mobile robot 100 via thenetwork 130 to the server 140, database 150, and/or remote platform 160shown in FIG. 11.

In some implementations, the location of the detected human as anon-human and/or the location of the detected animal as a non-animal ona map of the area in a memory that is communicatively coupled to thecontroller based on a signal received by a communications interface ofthe mobile robot via a communications network or an input received froman interface of the mobile robot. For example, operation 16 maydetermine that there is a human and/or an animal present in the area,but operation 20 may determine that there is no human and/or animal(i.e., the object originally detected is human is non-human, and/or theobject originally detected as an animal is non-animal) in the area basedon the lack of a human and/or animal identification, motion detection,the heat, and/or sound signals generated by the at least one sensor. Thelocation of the non-human and/or non-animal in the area may be stored inthe fixed storage 120 (shown in FIG. 10) and/or the memory 118 of themobile robot 100, and/or at the server 140, database 150, and/or remoteplatform 160 as shown in FIG. 11. The location of the non-human and/ornon-animal may be determined by at least one of the sensor 102 and/or106 of the mobile robot 100.

FIG. 2 shows example operations in connection with the operation 20(shown in FIG. 1) of determining when the human and/or animal is presentin the area according to an implementation of the disclosed subjectmatter. That is, the determining whether there is at least one humanand/or animal identification, motion, heat, and/or sound within the areamay include operation 26, where a first signal may be output with afirst sensor of the at least one sensor, based on whether there is humanand/or animal identification, motion, heat, and/or sound within thearea. For example, sensor 102, sensor 102, and/or microphone 109 mayoutput a signal that may be used (e.g., by controller 114 shown in FIG.10 and/or server 140 and/or remote platform 160 shown in FIG. 11) todetermine whether there is human and/or animal identification, motion,heat, and/or sound within the area. The sensor 102, 106 may capture animage of a human and/or animal in the area, may detect motion within thearea, and/or may detect heat within the area and may output a signal.The microphone 109 may output a signal if there is noise within thearea.

At operation 30, a second signal may be output with a second sensor ofthe at least one sensor, based on whether there is human and/or animalidentification, motion, heat, or sound within the area. Similar tooperation 28, at least one of sensor 102, sensor 102, and/or microphone109 may output a signal that may be used (e.g., by controller 114 shownin FIG. 11 and/or server 140 and/or remote platform 160 shown in FIG.12) to determine whether there is human and/or animal identification,motion, heat, and/or sound within the area.

At operation 32, the controller 114 shown in FIG. 10 and/or server 140and/or remote platform 160 shown in FIG. 11 may determine whether thereis human and/or animal identification, motion, heat, and/or sound by thehuman and/or animal in the area based on the first signal and the secondsignal. That is, the first signal and second signal may be used toverify whether a human and/or an animal has been detected by the sensor102, sensor 106, and/or microphone 109. For example, if neither thefirst signal nor the second signal indicate that a human and/or ananimal is present in the area, operation 32 may determine that a humanand/or an animal is not present within the area. If only one of thefirst signal and the second signal indicate that a human and/or ananimal is present in the area, operation 32 may determine that the humanand/or an animal is not present. As a human and/or an animal is notpresent within the area, mobile robot may resume movement and outputtingUV light to disinfect the area. In some implementations, if only one ofthe first signal and second signal indicate that the human and/or animalis present, the controller 114 may control the light source to prohibitthe emission of UV light in order to increase safety. When both thefirst signal and the second signal indicate that a human and/or ananimal is present in the area, the controller 114 may control the lightsource to prohibit the emission of UV light.

FIG. 3 shows the example method 10 of FIG. 1 may include a method ofdetecting a door opening within the area and stopping the movement andthe UV light output by the robot based on the detection according to animplementation of the disclosed subject matter. At operation 34, anopening of a door within the area may be detected using the at least onesensor that is disposed on or separate from the mobile robot. Forexample, the sensor 102, 106 and/or the microphone 109 of the mobilerobot 100 may be used to detect the opening of a door within the area.Sensor 304 may be separate from the mobile robot and may be used todetect the opening of the door 302 as shown in FIG. 8. Sensor 304 may bean infrared sensor, a passive infrared sensor (PIR), a motion sensor, orthe like that may detect the motion of the door 302. Sensor 302 may becommunicatively coupled to the mobile robot 100 via the networkinterface 116 and communications network 130. The sensor 304 maytransmit a signal to the mobile robot when the door movement is detected(i.e., the door is being opened). At operation 36, the controller (e.g.,controller 114) may stop the movement of the mobile robot within thearea by controlling the drive system (e.g., drive system 108) andstopping the emission of the UV light by controlling the UV light sourcewhen the opening of the door is detected.

FIGS. 4-6 show a plurality of external views of a mobile robot 100 thatincludes sensors that may be used to detect humans and/or animals withinthe area and/or detect surfaces and objects in the area, and a lightsource to output UV light to disinfect the air, objects, and/or surfacesin the area according to implementations of the disclosed subjectmatter. The mobile robot 100 may include at least one sensor 102, 106(where sensor 102 may be shown as sensor 102 a and 102 b in FIG. 5), alight source 104 to output ultraviolet light, a drive system 108, a userinterface 110, and/or a stop button 112. A controller (e.g., controller114 shown in FIG. 12 and described below) may be communicatively coupledto the at least one sensor 102 (and/or sensor 106), the light source104, the drive system 108, the user interface 110 and the stop button112, may control the operations of the mobile robot 100.

The at least one sensor 102 (including sensors 102 a, 102 b shown inFIG. 5) may determine whether a human and/or an animal is within thearea, an orientation of the mobile robot 100 (e.g., a direction that afront side and/or a first side of a robot is facing), a location of themobile robot 100 (e.g., a location of the mobile robot 100 in an area),and/or when the light source 104 is within a predetermined distance of asurface and/or object in the area. In some implementations, the firstsensor 102 may detect air, a surface, and/or objects that maydisinfected with UV light from the light source 104.

In some implementations, the at least one first sensor 102 may have afield of view of 70 degrees diagonally. The at least one sensor 102 mayhave a detection distance of 0.2-4 meters. As shown in FIGS. 4-6, the atleast one first sensor 102 may be disposed over the light source 104.

The at least one first sensor 102 may include a first side sensordisposed on a first side of the mobile robot 100 and a second sidesensor that may be disposed on a second side of the device. For example,as shown in FIG. 5, sensor 102 a may be disposed on a first side (e.g.,a front side) of the mobile robot 100, and sensor 102 b may be disposedon a second side (e.g., a back side) of the mobile robot 100. Althoughsensors on two sides of the robot are shown in FIG. 6, there may be aplurality of sensors disposed on different sides of the mobile robot 102to at least detect humans, animals, surfaces, and/or objects. In someimplementations, sensor 102 a and/or sensor 102 b may be disposed overthe light source 104.

The light source 104 may be one or more bulbs, one or more lamps, and/oran array of light emitting diodes (LEDs) or organic light emittingdiodes (OLEDs) to emit UV light (e.g., light having a wavelength of 10nm-400 nm). The dosage of the UV light (e.g., intensity, duration,optical power output, or the like) may be controlled by the controller114, which may also turn on or off a portion or all of the devices(e.g., bulbs, lamps, LEDs, OLEDs) of the light source 104, for example,when a human and/or an animal is detected. The light source may becontrolled to emit UV light when the mobile robot is within an area, andmay be controlled to stop emitting light when a human and/or an animalis detected.

The at least one sensor 106 may be communicatively coupled to thecontroller 114 shown in FIG. 10, and may be used to detect whether ahuman and/or an animal is within the area. The sensor 106 may detectair, surfaces, and/or objects that may be mapped and/or disinfected withUV light from the light source 104. In some implementations, the atleast one sensor 106 may determine at least one of an orientation of themobile robot 100 (e.g., a direction that a front side and/or a firstside of a robot is facing), a location of the mobile robot 100 (e.g., alocation of the mobile robot 100 in an area), and/or when the lightsource 104 is within a predetermined distance of a surface and/or objectin the area.

In some implementations, the sensor 102, 106 may be a time-of-flightsensor, an ultrasonic sensor, a two-dimensional (2D) Light Detection andRanging (LiDAR) sensor, a three-dimensional (3D) LiDAR sensor, and/or aradar (radio detection and ranging) sensor, a stereo vision sensor, 3Dcamera, an image sensor, RGB (red green blue) camera, thermal camera,geometrical sensors, a microphone, a structured light camera, or thelike. The sensor 106 may have a field of view of 20-27 degrees. In someimplementations, the sensor 106 may have a detection distance of 0.05-4meters.

The mobile robot 100 may include a motor to drive the drive system 108to move the mobile robot in an area, such as a room, a building, or thelike. The drive system 108 may include wheels, which may be adjustableso that the drive system 108 may control the direction of the mobilerobot 100.

In some implementations, the mobile robot 100 may include a base withthe drive system 108, and the sensor 102, 106 may be disposed on thebase.

The controller 114 may control and/or operate the mobile robot 100 in anoperation mode which may be a manual mode, an autonomous mode, and/or atele-operation mode. In the manual mode, the controller 114 may receiveon or more control signals from the user interface 110 and/or the stopbutton 112. For example, a user may control the movement, direction,and/or stop the motion of the mobile robot 100 (e.g., when a humanand/or an animal is detected in the area) by making one or moreselections on the user interface 110. The stop button 112 may be anemergency stop (ESTOP) button which may stop all operations and/ormovement of the mobile robot 100 when selected. In some implementations,the controller 114 may receive at least one control signal via a networkinterface 116 (shown in FIG. 10) when operating when operating in thetele-operation mode. For example, the network interface may receivecontrol signals via network 130 from server 140, database 150, and/orremote platform 160, as described below in connection with FIG. 11.

In some implementations, when the mobile robot 100 is moving in adirection, the sensor 102, 106 may detect a geometry of a human, ananimal, one or more surfaces, objects, or the like. The output of thesensor 102, 106 may be, for example, a point cloud of a human, ananimal, and/or the one or more objects in the path of the mobile robot100. When the sensor 102 and/or sensor 106 is a stereo vision sensor,images from two sensors (i.e., where the two sensors may be part of thestereo vision sensor of the sensor 102 and/or sensor 106) within a knowndistance from one another distance may be captured at a predeterminedpoint in time, and/or at predetermined time intervals with a globalshutter. The global shutter may be configured so that the two sensors ofthe stereo vision sensor may capture images about simultaneously. One ormore features may be determined from the captured images, and becompared to one another to determine portions that are matching. As thefocal length of the two sensors of the stereo vision sensor and thedistance between the two sensors (e.g., about 6 cm) may be stored inmemory 118 and/or fixed storage 120 (shown in FIG. 10), the controller114 and/or the at least one first sensor 102 may use the captured imagesand the stored values to determine the distance from the sensor 102, 106to the surfaces and/or objects, and may be used by the processor foroutputting a dosage of UV light from the light source. In someimplementations, the sensor 102, 106 may include at least one laser,LED, and/or OLED, to radiate one or more points on surfaces of objects,when the objects may be without identifying features (e.g., blankwalls).

When detecting a human, an animal, a surface, and/or an object, thesensor 102, 106 may be a time-of-flight (TOF) sensor. At least onephoton of light may be output by the sensor 102, 106, and may betransmitted through the air. When the at least one photon of lightradiates the human, animal, surface, and/or object, a portion of thelight may be reflected by the human, animal, surface, and/or the objectmay return to a receiver portion of the sensor 102, 106. The sensor 106may calculate the time between sending the at least one photon of lightand receiving the reflection, and multiply this value by the speed oflight in air, to determine the distance between the sensor 102, 106 anda human, animal, surface, and/or object. This may be used to determinewhether a human and/or an animal is in the area, and/or generate the mapof the area that the mobile robot is operating within.

FIG. 10 shows example components of the mobile robot 100 suitable forproviding the implementations of the disclosed subject matter. Themobile robot 100 may include a bus 122 which interconnects majorcomponents of the mobile robot 100, such as the drive system 108, anetwork interface 116 operable to communicate with one or more remotedevices via a suitable network connection, the controller 114, a memory118 such as Random Access Memory (RAM), Read Only Memory (ROM), flashRAM, or the like, the stop button 112, the light source 104, the atleast one first sensor 102, a user interface 110 that may include one ormore controllers and associated user input devices such as a keyboard,touch screen, and the like, a fixed storage 120 such as a hard drive,flash storage, and the like, the at least one second sensor 106, amicrophone 103, and a speaker 107 to output an audio notification and/orother information.

The bus 122 allows data communication between the controller 114 and oneor more memory components, which may include RAM, ROM, and other memory,as previously noted. Typically RAM is the main memory into which anoperating system and application programs are loaded. A ROM or flashmemory component can contain, among other code, the Basic Input-Outputsystem (BIOS) which controls basic hardware operation such as theinteraction with peripheral components. Applications resident with themobile robot 100 are generally stored on and accessed via a computerreadable medium (e.g., fixed storage 120), such as a solid state drive,hard disk drive, an optical drive, solid state drive, or other storagemedium.

The network interface 116 may provide a direct connection to a remoteserver (e.g., server 140, database 150, and/or remote platform 160 shownin FIG. 13) via a wired or wireless connection (e.g., network 130 shownin FIG. 13). The network interface 116 may provide such connection usingany suitable technique and protocol as will be readily understood by oneof skill in the art, including digital cellular telephone, WiFi,Bluetooth®, near-field, and the like. For example, the network interface116 may allow the mobile robot 100 to communicate with other computersvia one or more local, wide-area, or other communication networks, asdescribed in further detail below. The mobile robot may transmit datavia the network interface to the remote server that may include whethera human and/or an animal is detected, a path of operation, the surfacesand/or areas radiated with UV light, and the like.

Many other devices or components (not shown) may be connected in asimilar manner. Conversely, all of the components shown in FIG. 10 neednot be present to practice the present disclosure. The components can beinterconnected in different ways from that shown. Code to implement thepresent disclosure can be stored in computer-readable storage media suchas one or more of the memory 118, fixed storage 120, or on a remotestorage location.

FIG. 11 shows an example network arrangement according to animplementation of the disclosed subject matter. Mobile robot 100described above, and/or a similar mobile robot 200 may connect to otherdevices via network 130. The network 130 may be a local network,wide-area network, the Internet, or any other suitable communicationnetwork or networks, and may be implemented on any suitable platformincluding wired and/or wireless networks. The mobile robot 100 and/ormobile robot 200 may communicate with one another, and/or maycommunicate with one or more remote devices, such as server 140,database 150, and/or remote platform 160. The remote devices may bedirectly accessible by the mobile robot 100, 200 or one or more otherdevices may provide intermediary access such as where a server 140provides access to resources stored in a database 150. The mobile robot100, 200 may access remote platform 160 or services provided by remoteplatform 160 such as cloud computing arrangements and services. Theremote platform 160 may include one or more servers 140 and/or databases150.

More generally, various implementations of the presently disclosedsubject matter may include or be embodied in the form ofcomputer-implemented processes and apparatuses for practicing thoseprocesses. Implementations also may be embodied in the form of acomputer program product having computer program code containinginstructions embodied in non-transitory and/or tangible media, such assolid state drives, DVDs, CD-ROMs, hard drives, USB (universal serialbus) drives, or any other machine readable storage medium, such thatwhen the computer program code is loaded into and executed by acomputer, the computer becomes an apparatus for practicingimplementations of the disclosed subject matter. Implementations alsomay be embodied in the form of computer program code, for example,whether stored in a storage medium, loaded into and/or executed by acomputer, or transmitted over some transmission medium, such as overelectrical wiring or cabling, through fiber optics, or viaelectromagnetic radiation, such that when the computer program code isloaded into and executed by a computer, the computer becomes anapparatus for practicing implementations of the disclosed subjectmatter. When implemented on a general-purpose microprocessor, thecomputer program code segments configure the microprocessor to createspecific logic circuits.

In some configurations, a set of computer-readable instructions storedon a computer-readable storage medium may be implemented by ageneral-purpose processor, which may transform the general-purposeprocessor or a device containing the general-purpose processor into aspecial-purpose device configured to implement or carry out theinstructions. Implementations may include using hardware that has aprocessor, such as a general purpose microprocessor and/or anApplication Specific Integrated Circuit (ASIC) that embodies all or partof the techniques according to implementations of the disclosed subjectmatter in hardware and/or firmware. The processor may be coupled tomemory, such as RAM, ROM, flash memory, a hard disk or any other devicecapable of storing electronic information. The memory may storeinstructions adapted to be executed by the processor to perform thetechniques according to implementations of the disclosed subject matter.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific implementations. However, theillustrative discussions above are not intended to be exhaustive or tolimit implementations of the disclosed subject matter to the preciseforms disclosed. Many modifications and variations are possible in viewof the above teachings. The implementations were chosen and described inorder to explain the principles of implementations of the disclosedsubject matter and their practical applications, to thereby enableothers skilled in the art to utilize those implementations as well asvarious implementations with various modifications as may be suited tothe particular use contemplated.

1. A method comprising: moving, using a drive system, a mobile robotwithin an area; emitting, using a light source of the mobile robot,ultraviolet (UV) light to disinfect at least a portion of the area;while emitting the UV light, determining whether there is at least oneof the group consisting of: a human, and an animal within the area usingat least one sensor; stopping, using a controller, movement of themobile robot within the area by controlling the drive system andstopping the emission of the UV light by controlling the UV light sourcewhen the at least one of the human and the animal is determined to bewithin the area; determining, using the at least one sensor, whetherthere is at least one selected from the group consisting of: humanidentification, animal identification, motion, heat, and sound withinthe area for a predetermined period of time; when the at least onesensor determines that there is no human identification, animalidentification motion, heat, and sound within the predetermined periodof time, controlling the light source to emit UV light and controllingthe drive system to move the mobile robot within the area; and when theat least one sensor determines that there is at least one of the groupconsisting of: human identification, animal identification, motion,heat, and sound within the predetermined period of time, controlling thelight source to prohibit the emission of UV light.
 2. The method ofclaim 1, further comprising: outputting a notification that the at leastone of the human and the animal is determined to be in the area, themovement of the mobile robot is being stopped, and the emission of theUV light from the light source is stopped.
 3. The method of claim 2,wherein the outputting the notification comprises: outputting thenotification selected from at least one of the group consisting of: anaudible notification, and a visual notification.
 4. The method of claim2, wherein the outputting comprises: transmitting, using acommunications interface, the notification via a communications network.5. The method of claim 1, further comprising: storing at least one ofthe location of the detected human as a non-human and the location ofthe detected animal as a non-animal on a map of the area in a memorythat is communicatively coupled to the controller based on a signalreceived by a communications interface of the mobile robot via acommunications network or an input received from an interface of themobile robot.
 6. The method of claim 1, wherein the determining whetherthe human is within the area comprises: capturing an image, using atleast one sensor, to determine whether the at least one of the human andthe animal is within the area.
 7. The method of claim 1, wherein thedetermining whether the at least one of the human and the animal iswithin the area comprises: capturing a thermal image, using a thermalsensor of the at least one sensor, to determine whether the at least oneof the human and the animal is within the area.
 8. The method of claim1, wherein the determining whether there is at least one selected fromthe group consisting of: human identification, animal identification,motion, heat, and sound within the area further comprises: outputting afirst signal, with a first sensor of the at least one sensor, based onwhether there is human identification, animal identification, motion,heat, or sound within the area; outputting a second signal, with asecond sensor of the at least one sensor, based on whether there ishuman identification, animal identification, motion, heat, or soundwithin the area; and determining whether there is human identification,animal identification, motion, heat, or sound by the at least one of thehuman and the animal in the area based on the first signal and thesecond signal.
 9. The method of claim 1, further comprising: detecting,using the at least one sensor that is disposed on or separate from themobile robot, an opening of a door within the area; and stopping, usingthe controller, movement of the mobile robot within the area bycontrolling the drive system and stopping the emission of the UV lightby controlling the UV light source when the opening of the door isdetected.